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SUMMARY 


The Langley Research Center has undertaken a technology developnent 
activity to provide a capability, the Miaalon Oriented Tenalnal Area 
Simulation (MOTAS). wherein terminal area and aircraft systems studies can be 
parlormed. An experiment was conducted to evaluate state-of-the-art voice 
recognition technology and specifically, the Threshold 600 Voice Recognition 
System to serve as an aircraft control Input device for the MOTAS Pseudo Pilot 
Station function. The results of the experiment using ten subjects showed a 
recognition error of 3.67 percent for a 48-word vocabulary tested against a 
programmed vocabulary of 103 words. After the ten subjects retrained the 
Threshold 600 system for the words which were rdsrecognired or rejected, the 
recognition error decreased to 1.96 percent. The rejection rates 
cases were less than 0.70 percent. Based on the results of the experiment, 
voice recognition technology and specifically the Threshold 600 Voice 
Recognition System were chosen to fulfill this MOTAS function. 





INTRODUCTION 


The Langley Research Center (LaRC) has undertaken a technology 
development activity to provide a capability, the Mission Oriented Terminal 
Area Simulation (MOTAS), wherein terminal area and aircraft systems studies 
can be conducted. MOTAS is a flexible, comprehensive simulation of the 
airborne, ground~based, and communication aspects of the terminal area 
environment. The airborne aspects will include advanced flight deck 
simulators such as the Terminal Configured Vehicle (TCV) simulator (ref. 1) 
and the Advanced Concepts Simulator, a full workload DC~9 simulator, general 
aviation simulators (business jet and light twin classes), computer-generated 
aircraft with simplified dynamics flown either completely by the computer or 
^jji^ough the use of pseudo pilot stations, and LaRC s TCV B— 737 aircraft flying 
at the Wallops Flight Center. The ground-based aspects of MOTAS Include air 
traffic control (ATC) techniques, such as metering and spacing algorithms or 
YBctoring algorithms, control options (speed control, alternate paths, etc), 
IFR separation standards, navigational aids, terminal area geometries and air- 
route structuring, runway handling constraints, and surveillance errors. The 
communications aspects reflect communications by live ATC controllers and 
aircraft crews, and communications by computer modelled ATC controllers and 
aircraft crews. In the cases of the computer modelled ATC controllers and 
aircraft crews, the communications include such aspects as message content, 
delays associated with message delivery, delays associated with workload, and 
priority delivery of messages. 
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This paper deals specifically with the evaluation ol technology and 
equipment obtained to perform the function of the MOTAS Pseudo Pilot 
Stations. The Pseudo Pilot Stations are used to make control inputs tu 
computer-generated aircraft in the MOTAS simulation. These aircraft have 
simplified flight dynamics, and the control Inputs consist of commands such as 
TURN RIGHT, TURN LKKT. HEADING, ALTITUDE, AIRSPEED, TURN DIRECT TO, and so 
forth, followed by numerical values or waypoint names. In deciding on the 
type of equipment to be used for this function, several consultations were 
held with personnel at the Federal Aviation Administration (FAA) Technical 
Center. The FAA Technical Center operates a simulation facility (ref. 2) 
similar in concept, although much larger, to the MOTAS simulation. One part 
of the FAA facility is made up of 64 Pseudo Pilot Stations (fig. D, any one 
of which can handle between five and eight aircraft simultaneously depending 
on maneuvering conditions. At present, the MOTAS configuration plan calls for 
four Pseudo Pilot Stations, one for each control sector which has been 
modelled. The FAA Technical Center's Pseudo Pilot Stations are made up of a 
special purpose keyboard and CRT display (fig. 2). It became apparent early 
on in the discussions with FAA Technical Center personnel, that LaRC personnel 
would be faced with training and skill retention problems if this type of 
equipment and procedures were used. This type of equipment would require 
learning code words representing waypoints, commands, and so forth, learning 
the nonstandard keyboard, and learning data input procedures. FAA experience 
showed that it took some time to learn the system and that practice was 
required to maintain the skills learned. This would be a definite problem for 
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the MOTAS Pseudo Pilot Stations since there would be extended breaks of time 
between MOTAS studies, and thus, a high probability of forgetting and losing 
skills acquired in training. 

Toward the end of these consultations, a possible alternate method was 
suggested. This method made use of voice recognition technology which was 
under study (ref. 3) at the FAA Technical Center by another researcher. The 
goal of the study was "to assess the state-of-the-art in recognition of the 
spoken word by means of computer technology in order to evaluate its potential 
usefulness in operational air traffic control". Here the researcher's problem 
was that "the keyboard language that must presently be used to communicate 
these data to the computer system is artificial, encoded, almost absolutely 
inflexible, difficult to learn and remember, subject to error, and a source of 
distraction to the user". This is exactly the situation encountered with the 
MOTAS Pseudo Pilot Station concept. The FAA's research indicated that the 
state-of-the-art in voice recognition technology had advanced to where it 
might be used to solve this type of problem. Further discussions with the FAA 
resulted in a recommendation to LaRC to acquire a specific voice recognition 
device for evaluation which appeared to meet the MOTAS requirements and was 
still reasonable in cost. The FAA had extensively tested an earlier model of 
the recommended voice recognition device. 

EQUIPMENT DESCRIPTION 

In May 1980, LaRC took delivery of a Threshold 600 Voice Recognition 
System (refs. 4, 5, and 6) manufactured by Threshold Technology, Inc., of 
Delran, New Jersey. The following paragraph is the manufacturer s description 
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(ref. 4) of the subject system. 

The Threshold 600 system is an electronic speech recognition device which 
automatically recognizes spoken words or phrases. These words or phrases are 
referred to as utterances and may be up to two seconds in duration. The 
utterances must be separated from each other by a very short pause. This is 
referred to as isolated or discrete speech. The terminal will accept 
extremely short pauses between utterances; however, the parameters of the 
human speech production mechanism dictate that pauses between utterances will 
be typically no less than 0.1 seconds. The terminal accepts utterances as 
short as 0.1 seconds. Typical spoken utterances will range from 0.25 seconds 
to 1.0 seconds. The processing time required by the terminal to recognize a 
spoken utterance will vary depending upon the size of vocabulary Implemented 
in the hardware, but will typically be approximately 0.25 seconds. This 
processing will begin at the completion of an utterance, and the next 
utterance can begin before this processing is complete. A READY indicator 
light is provided to indicate when the terminal is ready to accept an 
utterance. 

The Threshold 600 system (fig. 3) delivered to LaRC consists of an analog 
speech preprocessor, an LSI-11 microcomputer and a digital Input/output 
interface, an Ann Arbor Model 400E CRT terminal with keyboard for input and 
display purposes, an operator console and microphone preamplifier, and a tape 
cartridge unit. The speech preprocessor, microcomputer, and all interfacing 
elements are contained in the main terminal unit. Details on programming, 
training, and operating the Threshold 600 system can be found in references 4, 

5, and 6. 



EXPERIMENT DESCRIPTION 


In order to test the concept of using a voice recognition device for 
Pseudo Pilot Station command Inputs, an experiment similar to experiment 
number 1 described in reference 3 was conducted. (For data collection and 
analysis purposes only, the Threshold 600 system was linked to a Control Data 
Corporation CYBER 175 computer.) In reference 3, the language chosen for 
testing was that used by the nonradar or flight data controller in enroute 
centers. Two reasons for selecting this language were that it was one of the 
more complex languages in use and that the total repertoire of possible 
messages was larger than that of any of the other languages used in the 
control of aircraft. For these reasons and the similarity to the type of 
language to be used in MOTAS applications, and to provide a means for 
comparison with reference 3, this language was also chosen by LaRC to be used 
in the evaluation of the Threshold 600 system. It should be noted that in the 
experiment documented by reference 3, an earlier Threshold Technology system 
known as the VIP-100 system was used. The entire vocabulary which makes up 
this language is found in Table 1. 

Training 

The Threshold Technology 600 system must be prepared to recognize spoken 
words. This preparation is accomplished by training the system to recognize 
each individual's speech pronunciation patterns through the use of a built-in 
training routine. Each individual speaks ten utterances of each word or 
in the vocabulary to allow optimization of the stored data for 
variations in speech pattern. During the training procedure, the Threshold 
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600 system prompts the individual for the word to be spoken and automatically 
sequences to the next word after the individual has spoken the word ten 
times. After the initial training for the evaluation, each individual spoke 
each word one time to determine if the Threshold 600 system would recognize 
the vocabulary. Any word misrecognized or rejected was retrained by the 
individual. This was done to help overcome word pattern optimization problems 
caused by nervousness, monotone speaking, low or high voice loudness level, 
and so forth; for example, problems which could be associated with confronting 
for the first time an unknown device (speaking to a computer) and unknown 
operating procedures. Once this training procedure was completed, the 
individuals were ready to begin the tests used for the evaluation of the 
system^ s performance. 

Testing 

A group of ten subjects was chosen to participate in the testing of the 
Threshold 600 system for recognition accuracy and word rejection frequency. 
This group was made up of six men and four women and their ages varied from 
the early twenties to the early fifties. The experiment was conducted in two 
phases: The first phase, which took place immediately after the initial 
training was completed, required that each subject randomly speak each of the 
tested words ten times during five different sessions spread over several 
weeks. In the second phase, each subject retrained any word which was 
misrecognized or rejected by the Threshold 600 system, and then was retested 
during two separate sessions to determine if any improvement in recognition 
and/or rejection rates had occurred. 



The testing procedure used in this experiment was somewhat different from 
that used in reference 3. In reference 3, the 103-word vocabulary (Table 1) 
was divided into several subvocabularies depending on the ATC function of the 
word. Three of these subvocabularies (digits, messages, and fixes; Table 2) 
were each tested independently, that is, each word was tested for recognition 
against only those words in the subvocabulary. The LaRC experiment involved 
testing each word of these three subvocabularies for recognition against the 
entire 103-word vocabulary. This resulted in a more difficult experiment 
since it increased the probability for more words to be misrecognized for the 
spoken word. In addition, the subjects in reference 3 were retrained as the 
testing progressed instead of waiting until the end of an experiment phase as 
was the case with the LaRC experiment. 

Initially, the environment chosen for the testing was very quiet. This 
was done in an effort to reproduce the conditions that would exist during 
MOTAS operations. Unfortunately, after the training was completed and the 
first phase of the experiment was begun, the environment became very noisy at 
times due to construction of a new aircraft simulator in an adjoining room. 
Intermittently, for the rest of the experiment, the subjects had to contend 
with constant background noise and periodic sharp noises caused by equipment 
being dropped, warning buzzers being tested, and so forth. While the 
background noise apparently did not affect the Threshold 600 system to any 
extent, the sharp noises had a definite Impact resulting in misrecognition or 
rejection of spoken words, and spurious recognition or rejection when no word 
was being spoken. This obviously had an effect on the accuracy level of the 
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test results. In the MOTAS operational environment, this type of problem 
should not occur, and thus one would expect better results. 

TEST RESULTS AND DISCUSSION 

The teet results and discussion will be brohen do«. into four subject 
areas: (1) group recognition accuracy and rejection rate, (2) Individual 

subject recognition accuracy and rejection rate. U) individual word 
recognition accuracy and rejection rate, and (4) word pair confusion data. 

Group Recognition Accuracy and Rejection Rate 
As stated previously, a random sequence was generated for the words 
contained in Table 2, so that each word appeared ten times in the sequence. 

This led to a total of 480 words spoken during each test session. The 480 
words were made up of 120 words from the digits subvocabulary, 150 words from 
the messages subvocabulary, and 210 words from the fires subvocabulary. 

Figures 4 and 5 present the total group word recognition error and rejection 
rates, respectively. The figures are based on results obtained from ten 
subjects performing five tests each after the initial training and two tests 

each after retraining. 

Figure 4 presents the recognition error data for the total 480 words and 
then separately for each subvocabulary. Maximum and minimum errors are shown 
tor each phase of tests along with the average error for the group. Notice 
that in all cases, the retraining reduced both the average errors and the 
maximum errors thus decreasing the range of errors. The total error decreased 
from 3.67 percent to 1.96 percent. The worst results tor the subvocabularies 
occurred with the digits group, which had a 5.67 percent error lor initial 


9 


training and a 2.58 percent error after retraining. The results after the 
retraining were considered acceptable for the MOTAS Pseudo Pilot Station 
function, especially when one takes into account that further refinement may 
be obtained through additional retraining of problem words. 

Figure 5 presents the rejection rate data for the total group. As with 
figure 4, the data are plotted for the total tested vocabulary and also for 
the three subvocabularies. Rejection rate data were taken using the following 
procedure: If a rejection occurred (the system could not match the word 

spoken with any word in the vocabulary), the word was respoken until a 
recognition, whether correct or incorrect, occurred. Each rejection for a 
word was counted, and then the total number of rejections for all words was 

compared to the total number of words for each vocabulary or subvocabulary. 

Thus, it was possible for more than one rejection to be counted as a single 

word was tested. Comparing figure 5 to figure 4, one can see that the 

rejection rate was much lower than the recognition error rate. The rejection 
rate was low initially, 0.68 percent, and was lowered only slightly to 0.49 
percent after retraining. The digits subvocabulary, again, showed the worst 
rate after initial training. With retraining, it fell to the same region as 
the other subvocabularies. 

Several factors were found from observation to contribute to the 
magnitude of the recognition error and rejection rates. These factors were: 
(1) environmental noise, (2) microphone position, (3) voice volume level, 

(4) voice peculiarities, (5) nervousness, and (6) colds. 

Environmental noise. - The environmental noise was discussed previously, 
but it is worth pointing out again that sharp noises seemed to cause the 
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Threshold 600 system problems while constant background noise did not. 

Microphone position. - The microphone position, as pointed out in 
references 3-6, proved to be a critical factor. If the microphone was placed 
too close to the mouth, it tended to pick up breathing noises from some of the 
subjects. When placed too far away or too far to the side of the mouth, it 
did not always pick up the spoken word or a good representation of the word, 
and it seemed to become more susceptible to extraneous noises. 

Voice volume level. - The voice volume or loudness also appeared to be 
critical. Several of the subjects tended to have very soft voices which 
resulted in the Threshold 600 system not obtaining a good enough 
representation of the word spoken to recognize it. Indeed, once or twice a 
word was spoken where no input occurred at all. It was as if no word had been 

spoken. 

Voice peculiarities and subject nervousness. - Voice peculiarities such as 
hissing sounds and lisping, and subject nervousness which manifested itself as 
soft voice levels, shaky voice, and frequent pausing and speaking of "eh" also 
caused recognition and rejection problems to varying degrees for several of 
the subjects. 

Colds.- Finally, one of the subjects developed a cold during the first 
phase of testing which did affect the results as would be expected; however, 
the recognition and rejection rates were still in the acceptable region, thus 
no effort was made to retrain any words for that subject for the duration of 
the cold as was done for subjects in reference 3. 
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Individual Subject Recognition Accuracy and Rejection Rate 
This section presents the results for recognition error and rejection 
rate for each individual subject for the total vocabulary and each of the 
three subvocabularies. In addition, group averages are given after the three 
worst subjects were removed (determined solely by the total vocabulary 
recognition error, since the rejection rates were much lower overall). This 
was done since there was a large difference between these three and the other 
subjects. Finally, the four best subjects (again determined by the same 
method) were chosen as if this would have been for a MOTAS study and the 

resulting group averages are presented. 

Table 3 presents the total vocabulary recognition error for each subject 
based on their initial training. The individual subjects' averages varied 
from 0.96 to 7.46 percent with the group's overall average being 3.67 
percent. The best subject (subject 9) had a range of scores from 0.21 to 1.25 
percent for the five tests, while the worst subject (subject 5) had a range of 
2.71 to 13.96 percent. When the three worst subjects (subjects 3, 5, and 7) 
were removed from the data base, the remaining seven subjects had an average 
recognition error of 2.25 percent. Finally, choosing the four best subjects 
(subjects 1, 2, 6, and 9) from the group of seven resulted in an average 
recognition error of 1.87 percent which was considered acceptable for the 
MOTAS application. 

Table 4 presents the results of the digits subvocabulary recognition 
error for each subject. From an overall group point of view, this 
subvocabulary had the worst recognition error, 5.67 percent, and the average 
errors for individual subjects ranged from 1.00 to 11.83 percent. The best 
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subject (9) had recognition errors that ranged from 0.00 to 2.50 percent for 
an average of 1.00 percent; the worst subject (5) had errors that ranged from 
2.50 to 21.67 percent for an average of 11.00 percent. Removing the three 
subjects (3, 5, 7) with the highest errors resulted in a group average of 3.19 
percent. Finally, using only the four best subjects (1, 2, 6, 9) the group 

average error reduced to 2.33 percent. 

Table 5 presents the results of the messages subvocabulary recognition 
error for each subject. The group average error for this subvocabulary was 
3.33 percent, and the individual subjects' average errors varied from 0.53 to 
9.07 percent. The best subject (9) had errors that ranged from 0.00 to 3.33 
percent for an average of 1.33 percent; the worst subject (5) had errors that 
ranged from 3.33 to 20.00 percent for an average of 9.07 percent. Removing 
the three worst subjects (3, 5, 7) resulted in a group average error of 2.57 
percent. The four best subjects (1, 2, 6, 9) yielded an average error of 2.37 

percent. 

Finally, Table 6 presents the results of the fixes sub vocabulary 
recognition error for each subject. From an overall point of view, this 
subvocabulary had the best recognition error, 2.76 percent, and the individual 
subjects' average errors ranged from 0.57 to 8.76 percent. The best subject 
(9) had errors that ranged from 0.00 to 0.95 percent with an average of 0.67 
percent; the worst subject (5) had errors that ranged from 2.38 to 6.19 
percent for an average of 4.29 percent. Once again, removing the three worst 
subjects (3, 5, 7) resulted in a group average error of 1.48 percent. 

Grouping the four best subjects (1, 2, 6, 9) reduced the group average error 
to 1.24 percent. 
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Tables 7 through 10 present the results tor the rejection rates tor each 
Individual subject based on their Initial training. The average group 
rejection rate for the total vocabulary (Table 7) was 0.68 percent with the 
worst indlvldua rate being 1.79 percent. All but two subjects (2, 5) 
registered a rejection rate of 0.75 percent or less. Thus, the rejection rate 
results were much better than the recognition error results and were 
considered acceptable for the MOTAS application. As with the recognition 
error results, the worst rejection rate occurred with the digits subvocahulary 
(Table 8)i however, this was caused by subject 2 who had a very high rejection 
rate as compared to the other subjects. «hen subject 2 was removed from the 
data base, the group digits overage rejection rate tell In line with the other 
subvocabularles. For the other two subvocabularles (Tables 9 and 10). subject 

5 was found to have the worst results. 

Table 11 presents the total vocabulary recognition error for each subject 

after retraining. By comparing this table with Table 3. one can see the 
improvement which occurred after retraining. The group average recognition 
error dropped from 3.67 to 1.96 percent. Notice, however, that for two 
subjects (8. 9). the recognition error Increased. In one case (9). there was 
a slight increase (0.96 to 1.35 percent) which was attributed to a recognition 
problem with a word that had not occurred In the Initial testing and thus had 
not been retrained. For the second subject (8). the recognition error more 
than doubled. The majority of this Increase was caused by two words; one 
which more than tripled In recognition error, and the second which almost 
doubled, thus Indicating that the retraining of these two words was 
unsuccessful. For the four best subjects (1. 2. 6. 9). the group average 
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recognition error after retraining was 1.07 percent, a substantial 
reduction. In addition to these four, a fifth subject (4) would have 
qualified after retraining under the original definition for best four and was 
in fact the best performing of the subjects after retraining with a 


recognition error of 0.31 percent. Substituting this subject for the worst 
performer of the original four subjects further reduces the four subject group 
recognition error to 0.81 percent. Tables 12 through 14 present the 
recognition error data for the three subvocabularies for each subject after 
retraining. Comparing to Tables 4 through 6, one sees that the recognition 
error for the group reduced from 5.67 to 2.58 percent for digits, from 3.33 to 
1.70 percent for messages, and from 2.76 to 1.79 percent for fixes. 


Tables 15 through 18 present the rejection 
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after retraining. As before, the rejection rate was lower than the 
recognition error. Some improvement did occur (0.68 to 0.49 percent for the 
total vocabulary); however, because the rejection rate was so low, a single 
rejection could cause the results to appear to fluctuate. For example, one 
rejection in one test equalled a 0.21 percent rejection rate for that test. 

Individual Word Recognition Accuracy and Rejection Rate 
This section presents the recognition error and rejection rate results 
for each of the individual words tested (Table 2) for the entire group of 
subjects. The data are presented for subjects tested after the initial 
training and after retraining. When comparing this data to the results of 
reference 3, the reader is reminded (see previous discussion of experiment 
differences) that the experiment in reference 3 was more restricted in 
vocabulary size which contributed to the increased performance of the voice 



recognition system and that retraining occurred as was required during testing 
and not as a separate test. 

Table 19 presents the results for the digits subvocabulary. For all 
words, there was an improvement in recognition error after the retraining was 
completed. A very large improvement occurred for the words FIVE (15.40 to 
3.50 percent) and SIX (10.60 to 3.50 percent). The words ONE and NINER showed 
only slight improvement in recognition error after retraining. At 6.00 
percent and 8.00 percent, respectively, these words were the only two of the 
entire subvocabulary with recognition errors that were over 3.50 percent. For 
rejection rate, three words increased slightly while all of the other words 

improved. 

Table 20 presents the results for the messages subvocabulary. Eleven of 
the fifteen words showed recognition error improvement after retraining. One 
additional word showed only a slight increase in recognition error (0.40 to 
0.50 percent). The words AMEND (11.80 to 3.50 percent), PRINTSTRIP (7.80 to 
0.50 percent), and REPORTALTITUDE (5.20 to 0.50 percent) showed the most 
improvement after retraining while the words CORRECTION (0.60 to 2.50 
percent), HANDOFF (3.80 to 5.00 percent), and TRANSMIT (5.80 to 7.00 percent) 
increased in recognition error. For rejection rate, ten words showed 
improvement, but five words showed worse rates of rejection. Notice for the 
word AMEND, there was an increase in rejection rate (1.60 to 6.50 percent), 
while at the same time a decrease in recognition error (11.80 to 3.50 
percent). Thus, some of the recognition errors appeared to become rejections 

after retraining. 
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Table 21 presents the results for the fixes subvocabulary. Sixteen of 
the words showed recognition error improvement after retraining. TOBYHANNA 
(4.20 to 0.50 percent), STILLWATER (6.20 to 2.00 percent), and BENTON (18.20 
to 7.50 percent) showed the largest improvements, while the word FREELAND 
(3.80 to 9.50 percent) showed the greatest increase in recognition error. All 
but three words showed a decrease in rejection rate. As with the recognition 
error, the word FREELAND showed the greatest rejection rate increase (1.60 to 

6.50 percent) after retraining. 

Word Pair Confusion 

This section presents data on the confusion between word pairs, that is, 
the words which were misrecognized for the spoken word. Tables 22 through 24 
present the results on the number of different words misrecognized for a given 
spoken word after initial training and after retraining. Tables 25 through 30 
present the individual spoken words along with the corresponding misrecognized 
words and the percentage of occurrence. For all three subvocabularies , the 
number of words misrecognized for a spoken word after retraining either 
decreased or in a few cases (5 words) remained the same; the lone exception 
was the word SELINGSGROVE which increased from one to two misrecognized 
words. Looking at Tables 25 through 30, one can see that a large number of 
the misrecognized words did not belong to the tested vocabulary (Table 2), but 
did belong to the total vocabulary (Table 1) that was programmed in the 
Threshold 600 system. For example (see Tables 22 and 25), 34 of the 62 
misrecognized words were not part of the tested vocabulary. It is conceivable 
that the number of misrecognized words would decrease (no new additional 



^srecognlzed words) If the total progra-aed .otabulary was reduced to only 
rhe tested vocabulary, however, the setup as used served the purpose of 
evaluating a worst case situation where the programmed vocabulary approached 


a 11 /a Tf nnutestsd words SiTQ rcfflovcd from 
the maximum number of words allowable. If nontestea wor 

Tables 22 through 24, the totals would change as follows; 

Table 22 - 62 would change to 28 for initial training; 28 


would change to 10 for retraining 

Table 23 - 77 would change to 35 for Initial training; 21 


would change to 8 for retraining 

Table 24 - 105 would change to 58 for initial training; 

39 would change to 22 for retraining. 


CONCLUDING REMARKS 

This paper has described the evaluation of voice recognition technology 
and specifically, the Threshold 600 Voice Recognition System for use In the 
MOTAS Pseudo Pilot Station function where the Threshold 600 system would act 

as an aircraft control input device. 

The experiment results showed the recognition error after Initial 
training to be 3.67 percent for ten subjects when testing a 48-word vocabulary 
against a 103-word programmed vocabulary. After the ten subjects had 
retrained words which had been misrecognlsed or rejected, the recognition 
error decreased to 1.96 percent. The rejection rates for both cases were 
found to be less than 0.70 percent. The recognition error was reduced even 
further when the four best subjects were chosen as if they were to operate the 
four MOTAS Pseudo Pilot Stations. It Is conceivable, based on other similar 
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studies, that the recognition error could also have been reduced if only the 
48 test words had been programmed; however, one of the parameters of the 
experiment was to use as large a programmed vocabulary as practical for the 

testing. 

There are several operational characteristics of MOTAS which would appear 
to provide additional means for reducing the recognition error: (1) a smaller 
operational vocabulary which at this time is defined as containing 44 words in 
the total vocabulary, (2) the vocabulary does not contain as many word pairs 
which appear to sound alike and thus would be susceptible to misrecognition; 
and with those word pairs that do occur, the flexibility exists to change one 
of the words and still achieve the desired results, (3) multiple retraining to 
continually reduce the number of problem words that an operator encounters, 
and (4) the MOTAS Pseudo Pilot Stations will be setup so that the operator can 
preview and edit the output from the voice recognition device before the input 
is made to the aircraft model. 

Thus, based on the above reasons and the acceptable results obtained from 
the described experiment, the decision was made to use voice recognition 
technology and specifically, the Threshold 600 Voice Recognition System to 
perform the MOTAS Pseudo Pilot Station function. 
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TABLE l.“ PROGRAMMED VOCABULARY 


ZERO 

ONE 

TWO 

THREE 

FOUR 

FIVE 

SIX 

SEVEN 

EIGHT 

NINER 

BACKSPACE 

GO 

AMEND 

CANCEL 

CORRECTION 

DEPARTURE 

DISCRETECODE 

READOUT 

ACCEPTHANDOFF 

HANDOFF 

DROPTRACK 

PRINT STRIP 

HOLD 

RELEASE 

REPORTALTITUDE 

WEATHER 

TRANSMIT 

TYPE 

QUALIFIER 
BEACONCODE 
SPEED 
FIX , 

TIME 

ALTITUDE 

IDENT 

WILLIAMSPORT 

SELINGSGROVE 

MILTON 

HAZELTON 

WILKE SB ARRE 

EASTTEXAS 

LAKEHENRY 

TOBYHANNA 

ALLENTOWN 

STILLWATER 

BENTON 


SWEETV ALLEY 

LOPEZ 

SNYDERS 

SLATINGTON 

WHITEHAVEN 

RESORT 

PENNWELL 

HUGUENOT 

SOLBERG 

FREELAND 

BOEING 

DOUGLAS 

LOCKHEED 

CONVAIR 

VICKERS 

NORD 

BRITISH 

GENERAL 

MILITARY 

DEHAVILLAND 

ALPHA 

BRAVO 

CHARLIE 

DELTA 

ECHO 

FOXTROT 

GOLF 

HOTEL 

INDIA 

JULIET 

KILO 

LIMA 

MIKE 

NOVEMBER 

OSCAR 

PAPA 

QUEBEC 

ROMEO 

SIERRA 

TANGO 

UNIFORM 

VICTOR 

WHISKEY 

XRAY 

YANKEE 

ZULU 


DISCRETE 

DISCRETEDME 

DME 

NONDISCRETE 

NONDISCRETEDME 

TRANSPONDER 

TRANSPONDERDME 

TACAN 

TACAN64 

TACANDISCRETE 

ERASE 


TABLE 2.- EXPERIMENT VOCABULARY 


a-DIGITS b-MESSAGES 


ZERO 

ONE 

TWO 

THREE 

FOUR 

FIVE 

SIX 

SEVEN 

EIGHT 

NINER 

backspace 

ERASE 


AMEND 

CANCEL 

CORRECTION 

departure 

DISCRETECODE 

READOUT 

ACCEPTHANDOFF 

HANDOFF 

DROPTRACK 

PRINT STRIP 

HOLD 

RELEASE 

REPORTALTITUDE 

WEATHER 

TRANSMIT 


c-FIXES 


WILLIAMSPORT 

SELINGSGROVE 

MILTON 

HAZELTON 

WILKE SB ARRE 

EASTTEXAS 

LAKEHENRY 

TOBYHANNA 

ALLENTOWN 

STILLWATER 

BENTON 

SWEETV ALLEY 

LOPEZ 

SNYDERS 

SLATINGTON 

WHITEHAVEN 

resort 

PENNWELL 

HUGUENOT 

SOLBERG 

FREELAND 



TABLE 3 


- SUBJECT TOTAL TEST VOCABULARY PERCENT RECOGNITION ERROR 
PERFORMANCE (INITIAL TRAINING). 


SUBJECT 

TEST 1 

TEST 2 

1 

2.50 

1.67 

2 

1.46 

2.71 

3 

6.88 

5.83 

4 

2.08 

2.08 

5 

2.71 

9.79 

6 

3.96 

2.71 

7 

7.92 

8.96 

8 

1.04 

1.25 

9 

0.21 

1.04 

10 

1.67 

2.50 


TEST 3 

TEST 4 

TEST 5 

TOTAL 

1.88 

2.50 

1.46 

2.00 

1.67 

2.92 

2.29 

2.21 

6.67 

9.79 

7.71 

7.38 

1.67 

2.50 

4.17 

2.50 

13.96 

6.46 

4.38 

7.46 

1.46 

1.04 

2.29 

2.29 

5.63 

5.42 

2.50 

6.08 

3.54 

2.71 

3.75 

2.46 

1.25 

1.04 

1.25 

0.96 

2.92 

4.58 

5.00 

3.33 


TOTAL 


3.67 



TABLE 4. 


SUBJECT DIGITS VOCABULARY PERCENT RECOGNITION ERROR 
PERFORMANCE (INITIAL TRAINING). 


SUBJECT 

TEST 1 

TEST 2 

1 

1.67 

0.83 

2 

4.17 

5.83 

3 

10.83 

9.17 

4 

4.17 

0.83 

5 

2.50 

19.17 

6 

2.50 

0.83 

7 

15.83 

24.17 

8 

3.33 

0.83 

9 

0.00 

1.67 

10 

1.67 

1.67 


TOTAL 


TEST 3 

TEST 4 

TEST 5 

TOTAL 

0.83 

0.83 

0.83 

1.00 

4.17 

9.17 

8.33 

6.33 

9.17 

15.83 

12.50 

11.50 

3.33 

0.83 

5.00 

2.83 

21.67 

5.83 

5.83 

11.00 

0.00 

0.00 

1.67 

1.00 

8.33 

3.33 

7.50 

11.83 

6.67 

6.67 

5.00 

4.50 

2.50 

0.00 

0.83 

1.00 

5.00 

5.83 

14.17 

5.67 




5.67 
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SUBJECT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
TOTAL 


TEST 1 

TEST 2 

3.33 

3.33 

0.67 

0.67 

2.67 

0.00 

1.33 

0.67 

3.33 

10.67 

6.67 

4.67 

4.00 

2.67 

0.00 

2.67 

0.00 

0.67 

4.00 

6.67 


test 3 

TEST 4 

1.33 

6.00 

0.00 

1.33 

3.33 

4.67 

1.33 

2.67 

20.00 

7.33 

4.00 

2.67 

3.33 

10.67 

1.33 

1.33 

0.67 

3.33 

4.67 

8.67 


TEST 5 

TOTAL 

1.33 

3.07 

0.00 

0.53 

0.00 

2.13 

3.33 

1.87 

4.00 

9.07 

4.67 

4.53 

0.00 

4.13 

1.33 

1.33 

2.00 

1.33 

2.67 

5.33 


3.33 
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TABLE 6. 


subject fixes vocabulary perceut eecogsition error performance 

(INITIAL TRAINING). 


SUBJECT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
TOTAL 


TEST 1 

2.38 
0.48 
7.62 

1.43 

2.38 

2.86 

6.19 
0.48 
0.48 
0.00 


TEST 2 
0.95 

2.38 

8.10 

3.81 

3.81 

2.38 

4.76 
0.48 
0.95 
0.00 


TEST 3 

2.86 

1.43 
7.62 
0.95 
5.24 
0.48 
5.71 
3.33 
0.95 
0.48 


TEST 4 
0.95 
0.48 
10.00 
3.33 

6.19 
0.48 

2.86 

1.43 
0.00 
0.95 


TEST 5 
1.90 
0.48 
10.48 
4.29 

3.81 
0.95 

1.43 

4.76 
0.95 

1.43 


TOTAL 

1.81 

1.05 

8.76 

2.76 
4.29 

1.43 

4.19 
2.10 
0.67 
0.57 

2.76 
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TABLE 7. - SUBJECT TOTAL TEST VOCABULARY PERCENT REJECT PERFORMANCE 

(INITIAL TRAINING). 


SUBJECT 

TEST 1 

TEST 2 

1 

0.42 

0.00 

2 

4.17 

0.83 

3 

1.46 

0.21 

4 

0.00 

0.00 

5 

2.71 

2.29 

6 

0.63 

0.21 

7 

0.00 

0.42 

8 

0.42 

0.42 

9 

0.83 

0.00 

10 

0.63 

0.00 

TOTAL 




TEST 3 

TEST 4 

TEST 5 

TOTAL 

0.42 

0.42 

0.21 

0.29 

1.67 

0.63 

1.25 

1.71 

0.00 

0.21 

1.46 

0.67 

0.00 

0.21 

0.00 

0.04 

1.25 

0.63 

2.08 

1.79 

0.83 

1.04 

0.00 

0.54 

0.21 

0.21 

0.00 

0.17 

0.42 

0.63 

0.42 

0.46 

0.42 

0.42 

0.21 

0.38 

0.00 

1.88 

1.25 

0.75 




0.68 


TABLE 8. - SUBJECT DIGITS VOCABULARY PERCENT REJECT PERFORMANCE (INITIAL 

TRAINING) . 


SUBJECT 

TEST 1 

TEST 2 

1 

0.00 

0.00 

2 

14.17 

3.33 

3 

0.00 

0.00 

4 

0.00 

0.00 

5 

1.67 

0.00 

6 

0.00 

0.00 

7 

0.00 

1.67 

8 

0.83 

0.83 

9 

0.00 

0.00 

10 

0.83 

0.00 

TOTAL 




TEST 3 

TEST 4 

TEST 5 

TOTAL 

0.00 

0.83 

0.00 

0.17 

4.17 

1.67 

5.00 

5.67 

0.00 

0.83 

3.33 

0.83 

0.00 

0.00 

0.00 

0.00 

0.83 

0.83 

0.00 

0.67 

0.00 

0.00 

0.00 

0.00 

0.83 

0.00 

0.00 

0.50 

0.83 

0.00 

0.00 

0.50 

1.67 

0.00 

0.00 

0.33 

0.00 

3.33 

2.50 

1.33 




1.00 
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TABLE 9. - SUBJECT MESSAGES VOCABULARY PERCENT REJECT PERFORMANCE 

(INITIAL TRAINING). 


SUBJECT ■ 

TEST 1 

TEST 2 

1 

0.00 

0.00 

2 

0.00 

0.00 

3 

0.00 

0.00 

4 

0.00 

0.00 

5 

4.67 

1.33 

6 

1.33 

0.00 

7 

0.00 

0.00 

8 

0.67 

0.00 

9 

0.00 

0.00 

10 

0.00 

0.00 


TEST 3 

TEST 4 

TEST 5 

TOTAL 

1.33 

0.00 

0.67 

0.40 

0.67 

0.67 

0.00 

0.27 

0.00 

0.00 

0.67 

0.13 

0.00 

0.00 

0.00 

0.00 

0.67 

0.00 

1.33 

1.60 

0.67 

3.33 

0.00 

1.07 

0.00 

0.67 

0.00 

0.13 

0.00 

0.00 

0.00 

0.13 

0.00 

0.00 

0.00 

0.00 

0.00 

2.67 

0.00 

0.53 


TOTAL 


0.43 



TABLE 10 - SUBJECT EKES VOCABULARY PERCENT REJECT PERFORMANCE (INITIAL 

iABLC. lu. training). 


SUBJECT 

TEST 1 

TEST 2 

TEST 3 

TEST 4 

TEST 5 

TOTAL 

1 

0.95 

0.00 

0.00 

0.48 

0.00 

0.29 

2 

1.43 

0.00 

0.95 

0.00 

0.00 

0.48 

3 

3.33 

0.48 

0.00 

0.00 

0.95 

0.95 

4 

0.00 

0.00 

0.00 

0.48 

0.00 

0.10 

5 

1.90 

4.29 

1.90 

0.95 

3.81 

2.57 

6 

0.48 

0.48 

1.43 

0.00 

0.00 

0.48 

7 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

8 

0.00 

0.48 

0.48 

1.43 

0.95 

0.67 

9 

1.90 

0.00 

0.00 

0.95 

0.48 

0.67 

10 

0.95 

0.00 

0.00 

0.48 

1.43 

0.57 


TOTAL 


0.68 



TABLE 11. 


SUBJECT TOTAL TEST VOCABULARY PERCENT RECOGNITION 
ERROR PERFORMANCE (RETRAINED). 


SUBJECT 

TEST 1 

1 

0.83 

2 

0.63 

3 

2.29 

4 

0.42 

5 

2.29 

6 

1.04 

7 

2.29 

8 

5.00 

9 

2.08 

10 

2.08 

TOTAL 



TEST 2 

TOTAL 

1.67 

1.25 

0.21 

0.42 

1.04 

1.67 

0.21 

0.31 

5.21 

3.75 

1.46 

1.25 

1.88 

2.08 

5.42 

5.21 

0.63 

1.35 

2.50 

2.29 


1.96 



TABLE 12. 


SUBJECT DIGITS VOCABULARY PERCENT RECOGNITION 
ERROR PERFORMANCE (RETRAINED). 


CTTR TFT.T 

TEST 1 

TEST 2 

TOTAL 

OU D w u w X 

1 

1.67 

1.67 

1.67 

2 

0.00 

0.83 

0.42 

3 

6.67 

1.67 

4.17 

4 

1.67 

0.00 

0.83 

5 

1.67 

6.67 

4.17 

6 

0.00 

0.83 

0.42 

7 

4.17 

1.67 

2.92 

8 

5.83 

7.50 

6.67 

9 

1.67 

0.83 

1.25 

10 

0.83 

5.83 

3.33 




2.58 

TOTAL 
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TABLE 13. - SUBJECT MESSAGES VOCABULARY PERCENT RECOGNITION 
ERROR PERFORMANCE (RETRAINED). 


SUBJECT 

TEST 1 

TEST 2 

TOTAL 

1 

0.00 

1.33 

0.67 

2 

0.67 

0.00 

0.33 

3 

0.67 

1.33 

1.00 

4 

0.00 

0.00 

0.00 

5 

1.33 

8.00 

4.67 

6 

2.67 

3.33 

3.00 

7 

1.33 

0.00 

0.67 

8 

0.67 

2.67 

1.67 

9 

4.67 

0.67 

2.67 

10 

2.67 

2.00 

2.33 


TOTAL 


1.70 


TABLE 14. “ SUBJECT FIXES VOCABULARY PERCENT RECOGNITION 
ERROR PERFORMANCE (RETRAINED). 


SUBJECT 

TEST 1 

TEST 2 

TOTAL 

1 

0.95 

1.90 

1.43 

2 

0.95 

0.00 

0.48 

3 

0.95 

0.48 

0.71 

4 

0.00 

0.48 

0.24 

5 

3.33 

2.38 

2.86 

6 

0.48 

0.48 

0.48 

7 

1.90 

3.33 

2.62 

8 

7.62 

6.19 

6.90 

9 

0.48 

0.48 

0.48 

10 

2.38 

0.95 

1.67 

TOTAL 



1.79 
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TABLE 15. - SUBJECT TOTAL TEST VOCABULARY PERCENT REJECT 
PERFORMANCE (RETRAINED). 


SUBJECT 

TEST 1 

TEST 2 

TOTAL 

1 

0.00 

0.21 

0.10 

2 

0.21 

0.63 

0.42 

3 

0.21 

0.00 

0.10 

4 

0.21 

0.00 

0.10 

5 

2.71 

0.00 

1.35 

6 

0.63 

0.21 

0.42 

7 

0.00 

0.00 

0.00 

8 

2.50 

1.46 

1.98 

9 

0.00 

0.42 

0.21 

10 

0.21 

0.21 

0.21 

TOTAL 



0.49 


TABLE 16. 

- SUBJECT DIGITS VOCABULARY 
(RETRAINED). 

PERCENT 

reject performance 

SUBJECT 

TEST 1 

TEST 2 

TOTAL 

1 

0.00 

0.00 

0.00 

2 

0.00 

2.50 

1.25 

3 

0.83 

0.00 

0.42 

4 

0.00 

0.00 

0.00 

5 

0.00 

0.00 

0.00 

6 

0.83 

0.00 

0.42 

7 

0.00 

0.00 

0.00 

8 

2.50 

0.00 

1.25 

9 

0.00 

0.00 

0.00 

10 

0.83 

0.83 

0.83 


TOTAL 


0.42 



TABLE 17. - SUBJECT MESSAGES VOCABULARY PERCENT REJECT 
PERFORMANCE (RETRAINED). 


SUBJECT 

TEST 1 

TEST 2 

TOTAL 

1 

0.00 

0.00 

0.00 

2 

0.67 

0.00 

0.33 

3 

0.00 

0.00 

0.00 

4 

0.67 

0.00 

0.33 

5 

8.67 

0.00 

4.33 

6 

1.33 

0.00 

0.67 

7 

0.00 

0.00 

0.00 

8 

0.00 

0.00 

0.00 

9 

0.00 

1.33 

0.67 

10 

0.00 

0.00 

0.00 


TOTAL 


0.63 


TABLE 18. 

- SUBJECT FIXES VOCABULARY PERCENT 
(RETRAINED). 

REJECT PERFORMANCE 

SUBJECT 

TEST 1 

TEST 2 

TOTAL 

1 

0.00 

0.48 

0.24 

2 

0.00 

0.00 

0.00 

3 

0.00 

0.00 

0.00 

4 

0.00 

0.00 

0.00 

5 

0.00 

0.00 

0.00 

6 

0.00 

0.48 

0.24 

7 

0.00 

0.00 

0.00 

8 

4.29 

3.33 

3.81 

9 

0.00 

0.00 

0.00 

10 

0.00 

0.00 

0.00 

TOTAL 



0.43 


TABLE 19. - DIGITS TEST RESULTS. 


WORD 

INITIAL 

RETRAINED 

INITIAL 

RETRAINED 


PERCENT ERROR 

PERCENT ERROR 

PERCENT REJECT 

PERCENT REJECT 

ZERO 

2.00 

1.50 

0.00 

0.00 

ONE 

8.20 

6.00 

0.20 

0.00 

TWO 

4.20 

1.50 

0.60 

1.00 

THREE 

4.80 

2.00 

0.00 

0.00 

FOUR 

2.40 

1.00 

2.80 

0.50 

FIVE 

15.40 

3.50 

4.40 

0.50 

SIX 

10.60 

3.50 

1.80 

0.00 

SEVEN 

1.80 

1.00 

0.60 

0.50 

EIGHT 

6.40 

3.00 

1.40 

2.00 

NINER. 

9.40 

8.00 

0.00 

0.00 

BACKSPACE 

0.80 

0.00 

0.20 

0.50 

ERASE 

2.00 

0.00 

0.00 

0.00 

TOTAL 

5.67 

2.58 

1.00 

0.42 
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TABLE 20 


MESSAGES TEST RESULTS, 


WORD 

INITIAL 

RETRAINED 

INITIAL 

RETRAINED 

PF.RCENT ERROR 

PERCENT ERROR 

PERCENT REJECT 

PERCENT REJECT 

AMEND 

11.80 

3.50 

1.60 

6.50 

CANCEL 

0.80 

0.50 

0.20 

0.00 

CORRECTION 

0.60 

2.50 

0.60 

0.00 

DEPARTURE 

3.60 

1.00 

0.80 

0.00 

DISCRETECODE 

1.20 

0.00 

0.00 

0.50 

READOUT 

0.20 

0.00 

0.00 

0.00 

ACCEPTHANDOFF 

0.00 

0.00 

0.80 

0.00 

HANDOFF 

3.80 

5.00 

0.40 

0.50 

DROPTRACK 

1.40 

0.50 

0.20 

0.00 

PRINT STRIP 

7.80 

0.50 

0.40 

1.00 

HOLD 

0.60 

0.00 

0.00 

0.00 

RELEASE 

6.80 

4.00 

0.20 

0.00 

REPORTALTITUDE 

5.20 

0.50 

0.80 

0.00 

WEATHER 

0.40 

0.50 

0.20 

1.00 

TRANSMIT 

5.80 

7.00 

0.40 

0.00 

TOTAL 

3.33 

1.70 

0.43 

0.63 
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WORD 


WILLIAMSPORT 

SELINGSGROVE 

MILTON 

HAZELTON 

WILKE SB ARRE 

EASTTEXAS 

LAKEHENRY 

TOBYHANNA 

ALLENTOWN 

STILLWATER 

BENTON 

SWEETV ALLEY 

LOPEZ 

SNYDERS 

SLATINGTON 

WHITEHAVEN 

RESORT 

PENNWELL 

HUGUENOT 

SOLBERG 

FREELAND 

TOTAL 


TABLE 21. - FIXES TEST RESULTS. 


INITIAL 
PERCENT ERROR 

0.60 

0.20 

5.80 

0.80 

0.00 

0.60 

0.20 

4.20 

1.80 

6.20 

18.20 

1.20 

1.60 

1.60 

2.60 

2.80 
0.60 
0.40 : 

3.80 

1.00 

3.80 

2.76 


RETRAINED 
PERCENT ERROR 

0.50 

1.00 

7.50 
0.50 
0.00 
0.00 
0.50 
0.50 
0.00 

2.00 

7.50 
0.00 
0.00 

1.50 
0.50 
0.50 
0.00 
0.00 

4.50 

1.00 

9.50 

1.79 


INITIAL 

PERCENT REJECT 

0.20 

0.00 

2.80 

1.20 

0.60 

0.00 

0.20 

1.00 

2.20 

0.20 

2.60 

0.00 

0.20 

0.00 

0.00 

0.20 

0.20 

0.00 

0.40 

0.40 

1.60 

0.68 


RETRAINED 
PERCENT REJECT 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.50 

0.50 

0.00 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

6.50 

0.43 
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TABLE 22. - WORD CONFUSION SUMMARY, DIGITS. 


WORD 


ZERO 

ONE 

TWO 

THREE 

FOUR 

FIVE 

SIX 

SEVEN 

EIGHT 

NINER 

BACKSPACE 

ERASE 


NUMBER OF WORDS CONFUSED 


INITIAL 


RETRAINED 


5 1 
3 

6 
7 
3 
9 
2 
5 
5 
13 
2 
2 


TOTAL 


62 


28 




TABLE 23. - WORD CONFUSION SUMMARY, MESSAGES 


NUMBER OF WORDS CONFUSED 


WORD 


INITIAL RETRAINED 


AMEND 

CANCEL 4 
CORRECTION 3 
DEPARTURE 10 
DISCRETECODE 5 
READOUT 1 
ACCEPTHANDOFF 0 
HANDOFF 10 
DROPTRACK 5 
PRINT STRIP 6 
HOLD 2 
RELEASE 3 
REPORTALTITUDE 5 
WEATHER 2 
TRANSMIT 7 


3 

1 

3 
2 
0 
0 
0 

4 
1 
1 


1 

1 

1 


TOTAL 


77 


21 


O ro 


TABLE 24. - WORD CONFUSION SUMMARY, FIXES 


WORD 


WILLIAMSPORT 

SELINGSGROVE 

MILTON 

HAZELTON 

WILKE SB ARRE 

EASTTEXAS 

LAKEHENRY 

TOBYHANNA 

ALLENTOWN 

STILLWATER 

BENTON 

SWEETV ALLEY 

LOPEZ 

SNYDERS 

SLATINGTON 

WHITEHAVEN 

RESORT 

PENNWELL 

HUGUENOT 

SOLBERG 

FREELAND 


NTTMRRR OF WORDS CONFUSED 


INITIAL 


RETRAINED 


2 

1 

8 

3 

0 

3 

1 

5 

3 

4 

22 

5 

4 

6 
6 

5 
2 
2 

10 

3 

10 


1 

2 

5 

1 

0 

0 

1 

1 


105 


39 


TOTAL 


00N3^OO»— ‘H- *N300'^U>0 



TABLE 25. - WORD CONFUSION, DIGITS (INITIAL TRAINING). 


WORD WORD CONFUSED (PERCENT OF TOTAL CONFUSION) 


ZERO 

ONE 

TWO 

THREE 

FOUR 

FIVE 

SIX 

SEVEN 

EIGHT 

NINER 


BACKSPACE 

ERASE 


KIL0(40.0), SIERRA(30.0), DISCRETECODE(IO.O), SEVEN(IO.O), 
SWEETVALLEY(IO.O) 

NORD(63.4), BOEING(31.7), MIKE(4.9) 

SPEED(57.1), GENERAL(14.3), HAZELTON(14.3), DME(4.8), 
THREE(4.8), UNIFORM(4.8) 

DISCRETE(33.3), SPEED(33.3), SLATINGTON(12.5), SNYDERS(8.3) , 
BRITISH(4.2), CORRECT ION( 4.2), TWO(4.2) 

S0LBERG(50.0), NORD(25.0), STILLWATER(25.0) 

TYPE(44.2), TIME(22.1), SOLBERG(9.1) , CHARLIE(6.5) , MIKE(6.5), 
NINER(6.5), PAPA(2.6), LMA(1.3), STILLWATER(1.3) 

FIX(98.1), SLATINGTONa.9) 

BENTON(55.6), DME(ll.l), SWEETVALLEY( 1 1 . 1 ) , TYPE(ll.l), 
WHISKEY(ll.l) 

HAZELTON(56.3), SPEED(18.8), VICTOR(12.5) , FK(9.4), 
YANKEE(3.1) 

MILITARY(40.4), WEATHER(14 .9) , TIME(10.6), FREELAND ( 6.5) , 
MIKE(6.5), SNYDERS(4.3), VICTOR(4.3), ALTITUDE(2.1) , 
DISCRETE(2.1), DEPARTURE(2.1), SEVEN(2.1), SLATINGTON(2. 1) , 
TOBYHANNA(2.1) 

XRAY(75.0), SLATINGTON(25.0) 

RELEASE(80.0), THREE(20.0) 
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TABLE 26. - WORD CONFUSION, MESSAGES (INITIAL TRAINING). 


WORD 

AMEND 


CANCEL 

CORRECTION 

DEPARTURE 


DISCRETECODE 

READOUT 

ACCEPTHANDOFF 

HANDOFF 


DROPTRACK 

PRINT STRIP 

HOLD 

RELEASE 

REPORTALTITUDE 

WEATHER 

TRANSMIT 


WORD CONFUSED (PERCENT OF TOTAL CONFUSION) 

IDENT(64.4), TRANSMIT(8.5) , DISCRETEDME(3 .4) , FREELAND(3.4) , 
T0BYHANNA(3.4), VICKERS(3.4), ALTITUDE(1 .7), EIGHT(1.7), 
HOLD(1.7), INDIA(1.7), LOPEZ(1.7), MILITARY(1.7) , THREE(1.7), 
WHITEHAVEN(1.7) 

EIGHT(25.0), TACAN64(25.0), VICKERS(25 .0) , WILL IAMSPORT( 25.0) 
BRITISH(33.3), DEPARTURE(33.3), ERASE(33.3) 

WHITEHAVEN(27.8), ALTITUDE(16.7) , CONVAIR(ll.l), MIKE(ll.l), 
CORRECT ION( 5.6), NINER(5.6), NONDISCRETE(5.6), QUEBEC(5.6), 
TRANSPONDERDME(5.6), WEATHER(5.6) 

SWEETVALLEY(33.3), BEACONCODE(16.7), DISCRETE(16.7) , 
FREELAND(16.7), SPEED(16.7) 

HUGUEN0T( 100.0) 

NO WORDS CONFUSED 

PENNWELL(26.3), HUGUENOT (1 5.8) , NINER(15.8), KILO(10.5), 
ALLENTOWN(5.3), GOLF(5.3), LAKEHENRY(5.3), SWEETVALLEY(5.3) , 
TANGO(5.3), TRANSPONDER(5.3) 

FIVE(28.6), FOXTROT(28.6), MILITARY(14.3), NINER(14.3), 
QUEBEC(14.3) 

VICT0R(82.1), IDENT(5.1), TRANSMIT(5.1) , DISCRETE(2.6) , 
NONDISCRETE(2.6), XRAY(2.6) 

GOLF(66.7), BRAVO(33.3) 

BRITISH(76.5), ERASE(20.6), FIX(2.9) 

DEPARTURE (80.8), LOCKHEED ( 7.7 ) , MILITARY(3.9) , PENNWELL(3.9) , 
UNIFORM(3.9) 

NINER(50.0), WHISKEY(50.0) 

PRINTSTRIP(58.6), VICTOR(20.7), BENTON(6.9), CANCEL(3.5), 
FREELAND (3. 5), MILITARY(3.5) , TACAN(3.5) 
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TABLE 27. - WORD CONFUSION, FIXES (INITIAL TRAINING) 


WORD 


WORD CONFUSED (PERCENT OF TOTAL CONFUSION) 


WILLIAMSPORT 

SELINGSGROVE 

MILTON 

HAZELTON 
WILKE SB ARRE 
EASTTEXAS 
LAKEHENRY 
TOBYHANNA 

ALLENTOWN 

STILLWATER 

BENTON 


SWEETVALLEY 

LOPEZ 

SNYDERS 

SLATINGTON 

WHITEHAVEN 

RESORT 

PENNWELL 

HUGUENOT 


SOLBERG 

FREELAND 


RESORT(66.7), FREELAND (3 3. 3) 

SLATINGTON( 100.0) 

MILITARY(34.5), DELTA(24.1), DEHAVILLAND(13.8) , G0(10.3), 
NONDISCRETE(6.9), DEPARTURE (3. 5), IDENT(3.5), NORD(3.5) 
EIGHT(SO.O), DISCRETE (25.0), REPORTALTITUDE(25 .0) 

NO WORDS CONFUSED 

BACKSPACE(33.3), RELEASE(33.3) , XRAY(33.3) 

SEVEN( 100.0) 

FREELAND(52.4), SWEETVALLEY(23.8), AMEND(14.3), MILITARY(4 .8) , 
N0VEMBER(4.8) 

ALTITUDE (66.7), IDENT(22.2), NINER(ll.l) 

S0LBERG(87.1), N0RD(6.5), DELTA(3.2), DISCRETECODE(3.2) 
N0NDISCRETE(30.8), FIX(ll.O), YANKEE(8.8), EIGHT(6.6), 
HAZELTON(5.5), MILTON(5.5), TWO(5.5), TACANDISCRETE(3.3) , 
TRANSPONDER(3.3), AMEND(2.2), SNYDERS(2.2) , TACAN(2.2), 
VICKERS(2.2), VICTOR(2.2), BRITISH(l.l), CORRECT lON(l.l), 
DISCRETE(l.l), DME(l.l), ERASE(l.l), FREELAND (1.1), 

REPORTALT ITUDE( 1.1), SIX(l.l) 

DISCRETE(33.3), FREELAND(16.7) , HUGUEN0T(16.7) , READOUT(16.7) , 
SEVEN(16.7) 

BACKSPACE(37.5), MILITARY(37 .5) , H0LD(12.5), HOTEL(12.5) 
IDENT(25.0), TYPE(25.0), BACKSPACE(12.5), FREELAND(12.5), 
NINER(12.5), SWEETVALLEY(12.5) 

SNYDERS(23.1), THREE(23.1), DISCRETE(15.4), SIX(15.4), 
SPEED(15.4), DME(7.7) 

LAKEHENRY(50.0), IDENT(21.4), BACKSPACE(14.3) , LOCKHEED (7 .1) , 
TYPE(7.1) 

WILLIAMSPORT(66.7), THREE(33.3) 

FREELAND(50.0), HANDOFF(50.0) 

READ0UT(31.6), FREELAND (10. 5) , HAND0FF(10.5) , SIERRA(10.5) , 
SWEETVALLEY(10.5), BEACONCODE(5.3), BENTON(5.3), LIMA(5.3), 
SK(5.3), ZULU(5.3) 

N0RD(40.0), STILLWATER(40.0), CHARLIE(20.0) 

BENTON(26.3), THREE(15.8), AMEND(10.5), LIMA(10.5), 

ZULU(10.5), DISCRETEDME(5.3), KILO(5.3), SLATINGTON(5.3) , 
WILLIAMSPORT(5.3), ZERO(5.3) 
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TABLE 28. - WORD CONFUSION, DIGITS (RETRAINED). 


WORD 


WORD CONFUSED (PERCENT OF TOTAL CONFUSION) 


ZERO 

ONE 

TWO 

THREE 

FOUR 

FIVE 

SIX 

SEVEN 

EIGHT 

NINER 

BACKSPACE 

ERASE 


SIERRA( 100.0) 

B0EING(83.3), THREE(8.3), N0RD(8.3) 

FREELAND(33.3), LIMA(33.3), SPEED(33.3) 

SPEED(75.0), DISCRETE(25.0) 

HOLD( 100.0) 

TIME(42.9), TYPE(42.9), MIKE(14.3) 

FIX(IOO.O) 

FREELAND(50.0), SLATINGTON(50.0) 

THREE(33.3), TACANDISCRETE(16 .7) , TYPE(16.7), VICTOR(16.7) , 

YANKEE(16.7) ^ 

WEATHER(37 .5) , SNYDERS(25 .0) , VICTOR(12.5) , CONVAIR(6.3) , 
FREELAND(6.3), MILITARY(6.3), TWO(6.3) 

NO WORDS CONFUSED 
NO WORDS CONFUSED 
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TABLE 29. - WORD CONFUSION, MESSAGES (RETRAINED). 


WORD 

AMEND 

CANCEL 

CORRECTION 

DEPARTURE 

DISCRETECODE 

READOUT 

ACCEPTHANDOFF 

HANDOFF 

DROPTRACK 

PRINT STRIP 

HOLD 

RELEASE 

REPORTALTITUDE 

WEATHER 

TRANSMIT 


WORD CONFUSED (PERCENT OF TOTAL CONFUSION), 

IDENT(71.4), MILITARy(14.3), TRANSMIT(14.3 
UNIFORM( 100.0) 

QUEBEC(60.0), BRITISH(20.0), WHITEHAVEN(20 
CONVAIR(50.0), NORD(50.0) 

NO WORDS CONFUSED 
NO WORDS CONFUSED 
NO WORDS CONFUSED 

TANG0(40.0), HUGUENOT(30.0), CANCEL(20.0) , 
MILITARY( 100.0) 

TRANSMIT(IOO.O) 

NO WORDS CONFUSED 

ERASE(75.0), BRITISH(12.5) , FK(12.5) 
DEPARTURE( 100.0) 

BRAVO( 100.0) 

PRINT STRIP(IOO.O) 


0 ) 


GO(IO.O) 


TABLE 30. - WORD CONFUSION, FIXES (RETRAINED) 


WORD 

WORD CONFUSED (PERCENT OF TOTAL CONFUSION) 

WILLIAMSPORT 

SELINGSGROVE 

RE SORT (100.0) 

DELTA(50.0), MILITARY(50.0) 


MILTON 

DELTA(73.3), BENTON(6.7), DME(6.7), GO(6.7), NONDISCRETE (6. 7) 

HAZELTON 

EIGHT( 100.0) 

WILKE SBARRE 

NO WORDS CONFUSED 

EASTTEXAS 

NO WORDS CONFUSED 

LAKEHENRY 

WHITEHAVEN( 100.0) 

TOBYHANNA 

AMEND( 100.0) 

ALLENTOWN 

NO WORDS CONFUSED 

STILLWATER 

FOUR(50.0), N0RD(25.0), SOLBERG(25.0) 

BENTON 

HAZELTON(26.7), TWO(26.7), DISCRETE(20.0) , FREELAND ( 6.7 ) , 
TACAN(6.7), TRANSPONDERDME(6.7), WHISKEY(6.7) 

SWEETVALLEY 

NO WORDS CONFUSED 

LOPEZ 

NO WORDS CONFUSED 

SNYDERS 

IDENT(66.7), VICKERS(33.3) 

SLATINGTON 

SPEED( 100.0) 

WHITEHAVEN 

LAKEHENRY( 100.0) 

RESORT 

NO WORDS CONFUSED 

PENNWELL 

NO WORDS CONFUSED 

HUGUENOT 

SWEETVALLEY(44.4), ZERO(33.3), HAZELTON(ll . 1) , SIERRA(ll.l) 

SOLBERG 

FOUR(50.0), FREELAND(50.0) 

FREELAND 

THREE(36.8), SLATINGTON(15 .8) , TACAN(15.8), TOBYHANNA(10.5) , 
DISCRETECODE(5.3), LIMA(5.3), REPORTALTITUDE(5.3) , 
SWEETVALLEY(5.3) 
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